Physicians have a need for knowledge and information of the structure and condition of a patient's internal anatomy. More importantly, physicians have a need for such information which may be obtained, when possible, by noninvasive techniques, that is, non-surgically. These needs were first addressed by the use of x-rays or radiographs. In recent years, however, the x-ray machine has been supplemented in many situations by medical imaging techniques which yield information in orientations which are not easily imaged by traditional x-rays and in three dimensions (3D). These techniques include, for example, computed tomography, magnetic resonance imaging, positron emission tomography, tomographic gamma scintillation imaging, and ultrasound scanning.
Perhaps the best known technique of this type is computed tomography (CT), also known as computed axial tomography (CAT). With computed tomography, a complete three dimensional examination is made up of a sequence of two dimensional (2D) cross sections or "slices". Slice information is acquired by rotating a thin, fan shaped beam of x-rays about the long axis of the patient. Each slice is irradiated by its edges; the transmitted x-ray beams are captured with position sensitive radiation detectors and, thus, x-ray attenuation measurements are obtained from many different directions across the slice. The two dimensional images are reconstructed from these measurements by a mathematical procedure known as convolution and back projection. The output of the reconstruction procedure is an array of numbers (known as picture elements or pixels in 2D and volume element or voxels in 3D) representing the radiological density (x-ray attenuation) at points within the slice.
A recently developed imaging modality which does not employ x-rays is magnetic resonance imaging (MRI). This technique uses large magnetic fields to excite protons within the body through the phenomenon of nuclear magnetic resonance (NMR). The excited protons produce a radio frequency (RF) signal which can be position encoded. Three dimensional information can be built up slice by slice, as with x-ray CT. The two dimensional slices are reconstructed for viewing using mathematical procedures analogous to those used in x-ray CT. In MRI, the information associated with each voxel is a composite of proton density (essentially, the amount of water present) and so called T1 and T2 relaxation times, which are functions of physical and chemical structure.
Other three dimensional imaging techniques fall within the realm of nuclear medicine. The basic principle here is to introduce radioactive substances (radio pharmaceuticals) into the body, relying on their pharmacological properties for uptake into specific organs (for example, radioactive iodine can be used to label the thyroid). These radioactive substances produce radiation which may be measured with position sensitive detectors external to the body, known as gamma cameras. Two dimensional projected images (comparable to those obtained with conventional x-rays) can be generated with analog electronic circuitry. To obtain reliable three dimensional information, however, single photon emission computed tomography (SPECT) or positron emission tomography (PET) is employed, both of which rely on digital techniques.
SPECT systems make use of routine gamma radiation emitting radiopharmaceuticals combined with tomographic scanning techniques and may be thought of as a tomographic gamma scintillation imaging technique. PET systems, in contrast, employ annihilation coincidence detection to detect positron annihilation radiation from positron emitting radiopharmaceuticals. In both of these modalities, the information associated with each voxel is a measure of the concentration of the radiopharmaceutical at the corresponding volume increment within the patient. SPECT and PET differ from CT and MRI in that the images are fundamentally physiological rather than anatomical (although certain MRI studies have a physiological component). Thus, for example, nuclear medicine studies are used to demonstrate abnormal growth activity in bones which otherwise appear normal.
Another common imaging modality which yields three dimensional information in digital format is diagnostic ultrasound. This technique relies on the reflection of sound waves at interfaces within the body (e.g., between fluid and soft tissue) to generate echoes; the elapsed time between the transmission of a pulsed sound wave and the reception of its echo gives a measure of the distance to the interface between types of tissue while the amplitude of the echo gives a measure of the difference in the material characteristics at an interface. Conventional ultrasound images are formed as slices in a manner analogous to CT. Digital techniques are not needed to produce the images although almost all modern devices store the image in digital format to avoid electronic drift and to facilitate post processing.
A major drawback of each of the imaging devices described above is that images are produced which comprise two dimensional slices of the internal anatomical structures being observed. Physicians must then mentally "stack" an entire series of these two dimensional slices in order to infer the structure of the three dimensional objects under investigation. Many problems are inherent in such an approach.
First, the interpretation of a series of stacked, two dimensional images by a physician requires a great deal of specialized knowledge and skill. Secondly, such an approach is extremely time consuming. Thirdly, the approach is prone to inaccuracy.
What is clearly needed is a medical display device which produces a three dimensional representation of internal anatomical structures produced from a full series of stacked two dimensional slices of that structure. Even more desirable is a medical image display device which provides the physician or other observer with the ability to manipulate the object and its image interactively in real time such that the object may be viewed from various directions and in various modes in real time. By real time display is meant that the video display output should be updated at or near video rates of 30 frames per second. Provided there is minimal or no delay between operator action and the corresponding charge in the final image, this update rate would provide instantaneous perceptual feedback. It should be clear that such an interactive three dimensional display system permitting a physician to visualize and interact with a shaded three dimensional representation of an anatomical structure would greatly facilitate the examination of the structure in conjunction with medical research, clinical diagnoses, and the planning and execution of treatment and surgical procedures.
A number of three dimensional display systems for medical objects have been described in the literature, but none of these provide realistic shaded images at the full resolution of the input data with real time interactive capabilities.
Three dimensional medical data sets can be displayed in the following ways: the data can be organized into a sequence of reprojected views or slices; it has been proposed to create true three dimensional images in space; and so-called two and a half dimensional (2.5D) images can be generated by projecting objects or object onto a two dimensional screen with depth cues given by shading.
Many computed tomography and magnetic resonance imaging display systems provide facilities to work through a sequence of two dimensional slices fairly rapidly, so that a trained physician can create a mental impression of the three dimensional structure. On the other hand, only the original slices captured by the imaging apparatus can be rapidly displayed. Reslicing or reformatting the image data to generate new two dimensional slices without re-imaging the patient, referred to as multi-planar reconstruction or MPR, slows the display process considerably.
True three dimensional images can be created in space using several different approaches. In one approach, a varifocal mirror is used to view slice images which are sequentially displayed on a cathode ray tube (CRT) under computer control. The mirror surface is vibrated in synchronism with the updating of the CRT. Different images are seen as the mirror vibrates back and forth, giving a sense of depth. Another proposed approach employs a volume of a fluorescent gas, such as iodine-chlorine (I--Cl) vapor, which is excited to fluorescence by intersecting laser beams. The laser beams are scanned in a similar manner to the raster scanning of video monitors and television screens, except in a three dimensional sense.
The most familiar method of generating realistic images from a three dimensional scene is to project it onto a two dimensional screen and rely on motion parallax, projective geometry, shading, and hidden surface removal to create the illustion of depth. The result is similar to conventional television and motion pictures, which viewers readily intrepret as representing three dimensional scenes.
Ultrasound imaging is a particularly useful technique in terms of noninvasiveness, safety and convenience to the physician and patient, and real time imaging capability. It is generally held that the ultrasound energy causes no discomfort or injury to the patient. In fact, ultrasound imaging is routinely used for imaging developing fetuses. As far as convenience, the physician simply positions the ultrasound transducer element or element array on the patient by hand and observes a cathode ray tube display of the image. This contrasts with other imaging techniques such as computed tomography and magnetic resonance imaging in which the patient is positioned in a large machine, out of immediate touch with the physician.
A major problem with ultrasound imaging is the low quality of the images provided thereby. "B mode" ultrasound images are formated in a manner somewhat similar to video images, that is, as repeated frames formed of image lines. Each line may represent the timing and amplitude of echoes resulting from a single pulse or firing of an ultrasound transducer or transducer element of an ultrasound array. Increasing the ultrasound pulse repetition rate increases the amount of image information available and, thus, increases the resolution of the resulting image. However, the maximum pulse repetition rate is limited by the speed of ultrasound energy travelling through various types of tissue After an ultrasound transducer element is "fired" it must be switched to receive mode to wait for returning reflections. Otherwise, outgoing pulses would overlap returning pulses which would complicate measurement of the depth of reflecting surfaces. In general, the maximum usable pulse repetition rate is inversely related to the thickness or depth of the organ or body part to be imaged with larger parts, such as the abdomen, requiring relatively low pulse repetition rates while smaller parts, such as the eye, allow higher rates.
To an extent, a trade-off can be made between the number of lines per images and the frame repetition rate. This allows more lines per frame at the expense of frame rate for a given pulse repetition rate. However, decreasing the frame rate diminishes the ability to image motion in real time. The lower limit for the frame rate is the rate at which image flicker begins to occur, which adversely affects viewability of the image.
Another factor which affects image quality is the problem of differentiating various types of soft tissue. Reflections of ultrasound energy from interfaces between different kinds of materials are caused by differences in the acoustic impedances of the materials. An interface between soft tissue and bone is relatively easy to image because of their significantly different acoustic impedances. In contrast, most types soft tissue have acoustic impedances which are not significantly different from that of water. Thus, reflections from an interface between two different types of soft tissue will be relatively weak and might be masked by system noise.
Even when relatively good ultrasound images can be obtained, recorded, and played back, it is still sometimes difficult to integrate a plurality of image slices into a three dimensional picture of the body region being imaged.